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(57) ABSTRACT 

An object data processing apparatus for decoding N pieces 
of coded data (N=positive integer) obtained by compres- 
sively coding N pieces of object data which constitute 
individual data to be recorded or transmitted and have a 
hierarchical structure, for each object data. This apparatus 
includes hierarchical information extraction means for 
extracting hierarchical information showing the hierarchical 
relationship of the N pieces of object data, according to the 
coded data; and table creation means for creating, according 
to the hierarchical information, an object table on which the 
respective object data are correlated with coded data of the 
respective object data. Therefore, the apparatus can perform 
extraction, selection, or retrieval of coded data correspond- 
ing to a specific object at high speed, and this enables the 
user to edit or replace the object data in short time with high 
controllability. 
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OBJECT DATA PROCESSING APPARATUS, In the object coding method, scene data corresponding to 

OBJECT DATA RECORDING APPARATUS, the scene 120 are compressively (coded in units of the 

DATA STORAGE MEDIA, DATA STRUCTURE lowermost objects constituting the scene 120, In other 

FOR TRANSMISSION words, scene data corresponding to the scene 120 are 

5 compressively coded for each of the objects 121, 123, 124, 

RELD OF THE INVENTION 125, 127, 128 and 129. 

The present invention relates to object data processing FIG. 19 is a diagram for explaining a data structure for 

apparatus, object data recording apparatus, data storage transmitting coded data corresponding to the respective 

media, and data structure for transmission. More objects mentioned above, which is obtained by performing 
particularly, the invention relates to an apparatus for decod- 1° object coding to the scene data of the scene 120. 

ing compressed data, such as compressed digital video data, in FIG. 19, MEg shows a multiplexed bit stream having 

digital audio data, and program data, an apparatus for a prescribed formal, obtained by multiplexing coded data of 

selecting desired data from the compressed data, an appa- the respective objects and auxiliary data. This multiplexed 

ratus for recording the compressed data, a medium storing bit stream MEg is transmitted as coded data corresponding 

the compressed data, an apparatus for outputting the com- scene data. 

pressed data, and a data structure for transmitting the com- multiplexed bit stream MEg is partitioned into plural 

pressed data. packets in prescribed units, i.e., each packet having pre- 

BACKGROUND OF THE INVENTION scribed number of bytes, and coded data of the respective 

objects are allocated to the packets having their own values 

In recent years, with the progress in information com- (SLC=1, 2, . . . ) as logical channels (LQ. 

pression technology, a digital video/audio service providing t, , • *u u- 1 j u-. . l 

. r ^ j i -r ^- t_j- • 1 To be specific, m the multiplexed bit stream MEg shown 

video information and audio mformation by digital signals . t?t/-« in j j j j * ^ u- , ■ n * j * 

, , , ^ L J ^- u in FIG. 19, coded video data or object [11 is allocated to 

has been put to practical use for broadcasting media, such as i*ni jti/:u • 1 • ii. toT/^'> jj 

, . J , ^ ^ .^ A^7 packets Pa3 and Pao having a logical channel SLC=3, coded 

ground broadcastmg, sateUite broadcasting, and CATV. ^.^^ ^^^^ ^^.^^^ ^^J is allocated to packets Pa5 and Pa7 

Under the existing circumstances, as a compressive cod- having a logical channel SLC=4, and coded audio data of 

ing method for the next generation, an object codmg method ^^-^^^ ^ allocated to a packet 4 having a logical channel 

has attracted attention. This object coding method is not to SLC-5. information relating to the byte number of packet 

uniformly compress the whole image, i.e., video data cor- ^^en multiplexed, the logical channel LC of each packet, 

responding to a single image, but to compress video data ^nd the packet transmission order is aUocated as control 

corresponding to d single image in units of individual information to a packet having another logical channel (not 

objects constituting the image while paying attention to the shown) for transmission. 

contents o I ^ image. . , . The objects [1] and [2] are the background image 121 and 

When video data corresponding to a smgle image is ^^e moving object 122 shown in FIG. 18, respectively, and 
subjected to the compressive coding in object units, com- 35 j^e object [3] is the background audio 123 shown in FIG. 18. 

pressed (coded) video data IS separable corresponding to the , , , , . 

respective objects, whereby a specific object in the image „ ° multiplexed bitstream MEg allocated to thepacket 

can be extracted or replaced. °^ ^LC=1 is information relaung to a 

, , . . , . , scene composition method for regeneratmg the scene com- 

Meanwhile as a method of implementing a data trans- „j respective objects (composition stream), and 
mission format for making the best use of the object coding 40 allocated to the packet Pa2 of logical channel SLC=2 is 

method, a method of mul iplexing compressed video data, information shov^dng how the cod5i data of the respective 

audio data, and other digital data is discussed. ^.^^ multiplexed (stream association tabic). 

There is MPEG4 as an international Standard of a method ^ , , , ^ , . , . . 

of multiplexing data compressed by the object coding Accordmgly, when a plurality of coded data ob^^^^^ 

method aSO/IECJTCl/SC29WGllN1483, "System Work- 45 ^^^J^j ^^^^^g multiplexed and transmitted, with the 

ing Draft", November 1996). Hereinafter, a description is coded data of the respective objects, the composition streana 

given of the data multiplexing method based on MPEG 4 f lowing the stnicture of a scene composed of the objects a^ 

and a method for reproducing the multiplexed data, with stream associaUon table showing the correlation of the 

reference to figures transmitted streams (each stream being a series of coded 

™„ 10 - J- r 1 • - .t. L- J- data corresponding to each object) are transmitted simulta- 

FIG. 18 IS a diagram for explaining the object coding 50 neQ„oiv ^ ^ ^ 

method. In the figure, reference numeral 120 designates a t-t^ • j- r 1 • • 

scene (an image) in a scries of images obtained from video ^® ^ ^ diagram for explaming a scene descnption 

data with audio. This scene 120 is composed of a plurality according to the composition stream, illustrating a descrip- 

of objects (sub-images) making a hierarchical structure. To ^^^^ oorrespondmg to the single image (scene) 120 
be specific, the scene 120 is composed of three objects: a 55 

background image (background) 121, a moving object 122 I" the scene description SD according to the composition 

that moves in the background, and a background audio 123 stream, the image 120 is shown by Scene 140, and the fact 

attendant on the c background. The moving object 122 is lhat the image 120 shown by Scene 140 is composed of the 

composed of four objects: a first wheel 124, a second wheel background image 121, the moving object 122, and the 
125, a body 126, and a moving object audio 127 attendant 60 background audio 123 is shown by Video(l) 141, Node(l) 

on the moving object. Further, the object of body 126 is 1^2, and Audio(l) 143. respectively. Here, Scene 140, 

composed of two objects: a window 128 and the other part Vidco(l) 141, Node(l) 142, and Audio(l) 143 are descrip- 

129. In the hierarchical structure, the objects 121-123 tors describing the image 120, the background image 121, 

belong to the uppermost first layer LI, the objects 124-127 ihe moving object 122, and the background audio 123 shown 
belong to the second layer 12 lower than the first layer LI, 65 FIG. 18, respectively. 

and the objects 128 and 129 belong to the third layer L3 Further, in the scene description SD, the fact that the 

lower than the second layer L2. moving object 122 shown by Node(l) 142 is composed of 
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the first wheel 124, the second wheel 125, the body 126, and data and the logical channels corresponding to the respective 

the moving object audio 127 is shown by Video(2) 144, streams (stream association table). The reason is as follows. 

Video(3) 145, Node(2) 146, and Audio(2) 147, respectively, In order to realize exchange of the contents of streams 

which are descriptors corresponding to these objects. corresponding to the respective objects and to facilitate 

Further, the fact that the body 126 shown by Nod6(2) 146 5 interface between the multiplexed bit stream and applica- 

is composed of the window 128 and the other part 129 is tions treating this multiplexed bit stream without changing 

shown by Vid6o(4) 148 and Video(5) 149, respectively, the scene composition (i.e., the hierarchical structure of the 

which are descriptors corresponding to these objects. objects constituting a scene), the structure for multiplexing, 

Each of the descriptors is given a stream index (stream id) ^^^^^^ depends on the physical layer of the multiplexed bit 
for identifying a stream corresponding to coded data of each 1° ^^^^^^^ "^^^t be separated from mam mformation (coded 

object in the multiplexed bit stream MEg. To be specific, as ^^^^) deluded m the multiplexed bit stream, 

shown in FIG. 20, stream indices Sid=l--Sid=5 are given to However, the multiplexed bit stream format according to 

the descriptors 141-145, respectively, and stream indices the prior art has the following drawbacks. 

Sid«6, Sid==7, and Sid=8 are given to the descriptors 148, A great advantage of object coding resides in that it 

149, and 147, respectively. Sid is a specific number of each enables extraction of coded data of a specific object from the 

stream id, multiplexed bit stream, and retrieval of a specific object on 

Accordingly, it can be seen from the scene description SD the data base containing the multiplexed bit stream, 
according to the composition stream that a scene is com- However, in order to recognize coded data of individual 
posed of what kinds of objects. However, the scene descrip- objects from the multiplexed bit stream MEg of the above- 
tion SD according to the composition stream does not mentioned structure, a complicated procedure is required as 
describe how the coded data corresponding to the respective follows. For example, to recognize coded data of lower- 
objects are multiplexed in the actual multiplexed bit stream layer objects from plural objects having a hierarchical 
MEg. structure, initially, the scene description according to the 

FIG. 21 is a diagram for explaining the stream association composition stream included in the multiplexed bit stream 

table AT. MEg is interpreted to find an object corresponding to a node. 

The stream association table AT shows the relationship and a stream corresponding to a lower object being a 

between the stream corresponding to coded data of each component of the object (node) is specified. Then, the stream 

object (i.e., a series of coded data corresponding to each association table AT is interpreted and, according to the 
object) and the logical channel (LC) specifying each packet 3^ stream id of the specified stream, a logical channel LC 

which is the partition unit of coded data when multiplexed. corresponding to the stream id is found. Thereby, coded data 

To be specific, on this table AT, the stream indices (id) of the of the specified object can be extracted from the multiplexed 

respective streams, the logical channel values (LC) corre- t)it stream MEg. 

sponding to the respective streams, and the logical channel Furthermore, since the hierarchical relationship of the 

values (LC) corresponding to upper streams of the respec- streams corresponding to the respective objects can be seen 

tive streams are correlated with each other. Here, the logical from the stream association table AT, it is possible to 

channel, LC corresponding to the upper stream of the analogize coded data of a specific object according to the 

streams (Sid=l~3) corresponding to the objects 121-123 of stream association table AT alone, but this analogy takes 

the first layer LI corresponds to the logical channel LC time and is not reliable. 

(SLC=2) of the packet Pa2 to which the stream association That is, oil the stream association table AT, information 

table is allocated. relating to objects as nodes is not clearly defined. In 

Accordingly, with reference to this table AT, the logical addition, since this table AT does not show the type of stream 

channel LC corresponding to each stream and the logical corresponding to coded data (for example, whether a stream 

channel LC of its upper stream (host stream) can be speci- corresponds to video data or audio data), other information 
fied. 45 such as the composition stream should be referred to. 

As described above, since the stream indices (Sid) are Further, for each stream, only its upper stream is known 

added to the descriptors 141-145 and 147-149 of the from the table AT. So, it is impossible to uniquely know that 

respective objects in the scene description SD according to coded data of each object is composed of which stream, and 

the composition stream shown in FIG. 20, the respective interpretation takes time. 

objects can be identified by the stream indices (Sid) from the 50 For example, in the scene description SD according to the 

composition stream and, therefore, the composition stream composition stream shown in FIG. 20, although Node(2) 

can be correlated with the stream association table shown in 146 corresponding to the subject 126 exists, a stream cor- 

FIG, 21. responding to Node(2) does not exist. So, on the stream 

As described above, the multiplexed bit stream MEg association table AT shown in FIG. 21, an entry correspond- 
includes the composition stream and the stream association 55 ing to Node(2) (i.e., stream id, LC corresponding to the 

table together with the coded data corresponding to the stream, and LC corresponding to the stream's upper-layer 

respective objects. Therefore, when the coded data of the stream) does not exists. 

respective objects are reproduced by decoding according to Accordingly, in order to extract the object 126 corre - 

the multiplexed bit stream MEg, it is possible to extract or sponding to Node(2), initially, stream indices (id) corre- 
retrieve coded data of a specific object designated according 60 sponding to the lower-layer objects 128 and 129 of Node(2) 

to the composition stream and the stream association table. 146 must be decided on the basis of the scene description SD 

This enables, for example, edition of the objects 121 to 129 according to the composition stream (refer to FIG. 20) and, 

constituting the scene 120 on the reproduction end. thereafter, the logical channels (LC) of packets containing 

In the multiplexed bit stream format according to the prior the streams having the decided stream indices must be 
art object coding, the scene description is expressed as 65 defined on the stream association table AT (refer to FIG. 21). 

information (composition stream) separated from informa- Further, there is a case where coded data corresponding to 

tion relating to the multiplexed state of the respective coded plural objects are transmitted without being multiplexed in 
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a particular transmission medium, such as computer network 
(internet). In this case, the bit stream has a data structure 
including no logical channels, and does not include the 
stream association table. 

In this case, detection of a specific object from the bit 
stream is carried out by interpreting the hierarchical struc- 
ture of the objects on the basis of the scene description SD 
according to the composition stream. However, when the 
number of the objects increases considerably, it requires a lot 
of time to interpret the hierarchical structure of the objects 
on the basis of the composition stream, resulting in poor 
controllability in replacement or edition of objects in a 
scene. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide an object 
data processing apparatus that speedily performs extraction, 
selection, or retrieval of coded data corresponding to a 
specific object from coded data corresponding to plural 
objects, whereby the user can edit or replace object data in 
short time with high controllability. 

It is another object of the present invention to provide an 
object data recording apparatus that records coded data 
corresponding to plural objects so that coded data of a 
specific object among the plural objects can be extracted, 
selected, or retrieved easily and speedily. 

It is still another object of the present invention to provide 
a data storage medium containing coded data having a data 
structure, which data structure realizes simple and speedy 
extraction, selection, or retrieval of coded data of a specific 
object from coded data corresponding to plural objects. 

It is a further object of the present invention to provide a 
data structure for transmission that realizes simple and 
speedy extraction, selection, or retrieval of coded data of a 
specific object from coded data corresponding to plural 
objects. 

Other objects and advantages of the invention will 
become apparent from the detailed description that follows. 
The detailed description and specific embodiments 
described are provided only for illustration since various 
additions and modifications within the scope of the inven- 
tion will be apparent to those of skill in the art from the 
detailed description. 

According to a first aspect of the present invention, there 
is provided an object data processing apparatus for decoding 
N pieces of coded data (N=positive integer) obtained by 
compressively coding N pieces of object data which con- 
stitute individual data to be recorded or transmitted and have 
a hierarchical structure, for each object data. This apparatus 
includes hierarchical information extraction means for 
extracting hierarchical information showing the hierarchical 
relationship of the N pieces of object data, according to the 
coded data; and table creation means for creating, according 
to the hierarchical information, an object table on which the 
respective object data are correlated with coded data corre- 
sponding to the respective object data. Therefore, extraction, 
selection or retrieval of coded data of a specific object can 
be carried out easily and speedily, and this enables the user 
to edit or replace the object data in short time with high 
controllability. 

According to a second aspect of the present invention, 
there is provided an object data processing apparatus for 
decoding N pieces of coded data (N=«positive integer) 
obtained by compressively coding scene data corresponding 
to one scene, for each of N pieces of objects constituting the 
scene. This apparatus includes hierarchical information 
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extraction means for extracting hierarchical information 
showing the hierarchical relationship of the respective 
objects constituting the scene, according to the coded data; 
and table creation means for creating, according to the 
5 hierarchical information, an object table on which the 
respective objects are correlated with coded data corre- 
sponding to the respective objects. Therefore, on the decoder 
side, extraction, selection or retrieval of a specific object 
from plural objects (video and audio) constituting one scene 
can be carried out easily and speedily with reference to the 
object table. 

According to a third aspect of the present invention, in the 
object data processing apparatus according to the second 
aspect, the hierarchical information extraction means is 

J 5 constmcted so that it extracts priority information showing 
the priority order of the respective objects, according to the 
coded data, in addition to the hierarchical information; and 
the table creation means is constructed so that it creates, 
according to the hierarchical information and the priority 

20 information, an object table on which the respective objects 
are correlated with coded data corresponding to the respec- 
tive objects and the priority order of the respective objects 
are shown. Therefore, when the throughput of the decoding 
apparatus is low and the apparatus cannot decode coded data 

25 of all objects, only objects having priorities higher than a 
prescribed priority are subjected to decoding. 

According to a fourth aspect of the present invention, the 
object data processing apparatus according to the second 
aspect, further includes identification information detection 

3Q means for detecting identification information for identify- 
ing coded data of a specific object designated, with reference 
to the object table; and decoding means for extracting coded 
data of the specific object from the N pieces of coded data 
according to the identification information, and decoding the 

35 extracted coded data. Therefore, in the decoding apparatus, 
retrieval of an object specified by the user is facilitated. 

According to a fifth aspect of the present invention, there 
is provided an object data processing apparatus for decoding 
multiplexed data including N pieces of coded data 

40 (N=positive integer) obtained by compressively coding 
scene data corresponding to one scene, for each of N pieces 
of objects constituting the scene. This apparatus includes 
hierarchical information extraction means for extracting 
hierarchical information showing the hierarchical rclation- 

45 ship of the N pieces of objects constituting the scene, 
according to information showing the correlation of the 
respective coded data and included in the multiplexed data; 
and table creation means for creating, according to the 
hierarchical information, an object table on which the 

50 respective objects are correlated with coded data corre- 
sponding to the respective objects. Therefore, extraction, 
selection or retrieval of a specific object from plural objects 
(video and audio) constituting one scene can be carried out 
easily and speedily, on the basis of the multiplexed data, 

55 with reference to the object table. 

According to a sixth aspect of the present invention, in the 
object data processing apparatus according to the fifth 
aspect, the hierarchical information extraction means is 
constructed so that it extracts priority information showing 

60 the priority order of the respective objects, according to the 
multiplexed data, in addition to the hierarchical information; 
and the table creation means is constructed so that it creates, 
according to the hierarchical information and the priority 
information, an object table on which the respective objects 

65 are correlated with coded data corresponding to the respec- 
tive objects and the priority order of the respective objects 
are shown. Therefore, when the throughput of the decoding 
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apparatus is low and the apparatus cannot decode coded data This apparatus selects coded data of a specific object from 

of all objects, only objects having priorities higher than a the multiplexed data with reference to the object table and 

prescribed priority are subjected to decoding. outputting the selected coded data. Therefore, on the basis of 

According to a seventh aspect of the present invention, the the multiplexed data, selection of a specific object from 
object data processing apparatus according to the fifth aspect 5 plural objects (video and audio) constituting one scene can 

further includes identification information detection means be carried out easily and speedily. 

for detecting identification information for identifying coded According to an eleventh object of the present invention, 

data of a specific object designated, with reference to the ^^^^^ ^ .^^^ ^^.^^ ^^^^ ^ ^^^^^ 

obiect table; and decodmg means for extracting coded data : c . • j* j m % 

1.- 1 J J . J- * a data storage for stormg data, and recording N pieces of 

of the speafic object from the multiplexed data according to ^=positive integer) in the data storage, which 

the identification information, and decoding the extracted j j j * • j i_ 1 • 

jjj rrn r t-j j- . . • t r coQCQ data are obtamed by compressively codmg N pieces 

coded data. Therefore, in the decoding apparatus, retrieval of ^ » . » . ■ . . , j - -j i ^ ♦ * u 

an ob ect s ecified b the user is facilitated object data which constitute individual data to be 

" ^ ..... recorded or transmitted and have a hierarchical structure, for 

According to an eighth aspect of the present iDvenlion, ^^^^^ ^ ^^ -^^^^^^ hierarchical infor- 
there is provided an object dau processing apparatus for 15 ^^(ion extraction means for exUacting hierarchical infor- 

selectmg coded data of a specific object data from N pieces ^^^^^ ^^^^^j hierarchical relationship of the respec- 

of coded data (Nopositive integer) obtained by compres- ^. ^jj^^ according to the coded data; and table 

sively coding N pieces of object data which institute ^^^^jion means for creating, according to the hierarchical 

individual data to be recorded or transmitted and have a information, an object table on which the respective object 
hierarchical stnicture for each object data. Ttis apparatus 20 data are correlated with coded data corresponding to the 

includes hierarchica informaUon extraction means for j„pective object data. This apparatus records the N pieces 

extractmg hierarchical information showmg the hierarchical ^^j^ j,,^ ^.^ject table corresponding to these 

relationship of the N pieces of object data, according to the j^j^ j^,^ ^ Therefore, extraction, selec- 

codcd data; and table creation means for creatmg, according ^^j^^^^ „f j^t^ „f ^ object can be 

to the hierarchical information, an object table on which the 25 ^^^^j ^^-^ ^ jj, ^^^^ reference to the object 

respective object data are correlated with coded data corre- ^^^^^ 

spending to the respective object data. This apparatus selects ^ r t • • 

coded data of a specific object data from the N pieces of , According to a twelfth aspect of the present invention, 

coded data with reference to the object table and outputting "'"^ object data recotding apparatus having 

the selected coded data. Therefore, selection of coded data 30 ' data aged recording N pieces of 

of a specific object can be carried out easily and speedily, data (N-positive mteger) in the data storage, which 

and this enables the user to edit, replace, or retrieve the '^'^^'^ compressively coding scene data 

object data in short time with high controllability. corresponding to one scene, for each of N pieces of objects 

, . . rlL constituting the scene. This apparatus includes hierarchical 

According to a ninth aspect of the present invention, there . r 4- * ^- c ^ ^- u- w \ 

•jj t.-.j. • . r 1.- information extraction means for extracting hierarchical 

is provided an object data processing apparatus for selecting 35 . r u • »u u* u* 1 1 *■ u- e *u 

^ J , ^ „ L . . r ■ r j j j . information showing the hierarchical relationship or the 

coded data of a specific obiect from N pieces of coded date . . • . .-^ * .t. 

• . X u. • J u -1 J- respective objects constituting the scene, accordmg to the 

(N=positive integer) obtained by compressively coding 1 jj * j . i_i r j- 

. , c I- /xr • coded data; and table creation means tor creating, according 

scene data corresponding to one scene, for each of N pieces ... 1. ,. ,. r l- . . li l- l.i. 

^ ,. ^ ..f r«- * • 1 J to the hierarchical information, an object table on which the 

of obiects constituting the scene. This apparatus includes , . , 1 \ . j j j . 

? , - c . respective objects are correlated with coded data corre- 

hierarchical information extraction means for extracting 40 . .r. i_- . rm.- * j 

i_. t.- 1 • r 1. • .1- i_* 1 1 *• sponding to the respective objects. Irus apparatus records 

hierarchical information showing the hierarchical relation- xt • c j jj^ j i_- ^ ^^^ j 

^ ,. , . , the N pieces of coded data and the object table correspond- 

snip 01 the respective obiects constituting the scene, accord- . , f. j j j . • .t. ^ . . c .t. 

• . -1 J J . J . LI r mg to these coded data in the data storage. Therefore, on the 

ing to the coded data; and table creation means lor creatine, . j . . ^. 1 r 

J- 4 *u I.* u- 1 * r 4- u- 4 4 ui decoder side, extraction, selection or retrieval of a specific 
according to the hierarchical into rmation, an obiect table on ,. , 1 ; ✓ -j j j- \ .- 
. . . ,l u- . 1 / j -.u J J J . obiect from plural objects (video and audio) constituting one 
which the respective objects are correlated with coded data 45 u r j 1 j j-i -^u ^ * 

.. ^ , . . • . r™. , scene can be performed easily and speedily with reference to 

corresponding to the respective objects. This apparatus ob'ect table j f j 

selects coded data of a specific object from the N pieces of ^ 

coded data with reference to the object table and outputting According to a thu-tecnth aspect of the present invention, 

the selected coded data. Therefore, on the decoder side, ^^jj^^^ ^^^^ recording apparatus according to the 

selection of a specific object from plural objects (video and 50 ^^^^^^^ ^P^^^' ^^^^^ corresponding to the N pieces 

audio) constituting one scene can be carried out easily and of coded data is added to the N pieces of coded data when 

speedily. being recorded. Therefore, management of the recorded 

According to a tenth aspect of the present invention, there ^^^^^ facilitated, 

is provided an object data processing apparatus for selecting According to a fourteenth aspect of the present invention, 
coded data of a specific object from multiplexed data 55 the object data recording apparatus according to the 

including N pieces of coded data (N-positive integer) twelfth aspect, the object table corresponding to the N pieces 

obtained by compressively coding scene data corresponding of coded dau is separated from the N pieces of coded data 

to one scene, for each of N pieces of objects constituting the when being recorded. Therefore, regardless of the size of the 

scene. This apparatus includes hierarchical information coded data, recording of the object table to a storage medium 
extraction means for extracting hierarchical infomiation 60 is carried out with high reliability. 

showing the hierarchical relationship of the respective According to a fifteenth aspect of the present invention, 

objects constituting the scene, according to information there is provided an object data recording apparatus having 

showing the correlation of the respective coded data and a data storage for storing data, and recording multiplexed 

included in the multiplexed data; and table creation means data including N pieces of coded data (N=positive integer) 
for creating, according to the hierarchical information, an 65 in the data storage, which coded data are obtained by 

object table on which the respective objects are correlated compressively coding scene data corresponding to one 

with coded data corresponding to the respective objects. scene, for each of N pieces of objects constituting the scene. 
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This apparatus includes hierarchical information extraction Therefore, it is not necessary to create an object table on the 

means for extracting hierarchical information showing the decoder side, whereby edition, replacement or retrieval of 

hierarchical relationship of the respective objects constitut- object data can be performed by a simple structure, in short 

ing the scene, according to information showing the corre- time with high controllability. 

lation of the respective coded data and included in the 5 According to a twenty-first aspect of the present 

multiplexed data; and table creation means for creating, invention, there is provided an object data processing appa- 

according to the hierarchical information, an object table on ratios for outputting N pieces of coded data (N»positivc 

which the respective objects are correlated with coded data ^teger) obtamed by compressively codmg scene data cor- 

corresponding to the respective objects. This apparatus respondmg to one scene, for each of N pieces of objects 
records the multiplexed data and the object table corre- lO constituting the scene. This apparatus includes hierarchical 

spending to the multiplexed data in the data storage. information extraction means for extracting hierarchical 

Therefore, on the decoder side, extraction, selection or information showing the hierarchical relaUonship of the 

retrieval of a specific object from plural objects (video and respective objects constituting the scene, according to the 

audio) constituting one scene can be performed easily and ^^^^^ ^^^^'^ ^"'^ ^^^^^ ^^^^^i^" ^^^^^g' according 

speedily with reference to the object table. 15 hierarchical information, an object table on which the 

» , , . . respective obiects are correlated with coded data corre- 

According to a sixteenth aspect of the present invention, j- * *u *■ u- * t^.* * * * 

.1. L' .j. J- * J- ..u sponding to the respective objects. This apparatus outputs 

m the obiect data recording apparatus according to the vt • r j j j * * i.- u fL i_- * * i_i 

, . . J- . 7i- 1 the N pieces of coded data to which the object table 

fifteenth aspect, the obiect table corresponding to the mul- ^ ^ ^, j j j ^ • jj j -n. r 

.-1 J J f • J * .1- 1.- T J J * u u • corresponding to these coded data is added. Therefore, it IS 

tiplexed data is added to the multiplexed data when being . ^ ^ ^ , , u- * * ui j j 

jjrTM_i- . r.i- jji- .m not necessary to create an object table on the decoder side, 

recorded. Therefore, management of the recorded obiect . . , : * • i r u- . / j 

table is facilitated whereby edition, replacement or retneval of objects (video 

a e is aci 1 a e . audio) constituting one scene can be performed by a 

According to a seventeenth aspect of the present ^^^^^^ structure, in short time with high controllability, 

invention, in the object data recording apparatus according According to a twemy-second aspect of the present 

to the fifteenth aspect, the object table corresponding to the invention, there is provided an object data processing appa- 

multiplexed data is separated from^e multiplexed data ^^^^^ ^^^^^ ^^^^ ^^^^ 

when toeing recorded. TTierefore, regardless of the size of the ^ apparatus according to the twenty-first aspect. This 

multiplexed data recording of the object table in a storage ^^^^^^^^^ ^^^^^^^ ^^^^ separation means for separating the 

medium is earned out with high reliabihty. ^^-^^ ^^^^^ f^^^ ^^^p^^ ^^^^^ ^^^^^^^ ^^^^^ 

According to an eighteenth aspect of the present for storing the separated object table. In this apparatus, 

invention, there is provided a data storage medium contain- decoding of the coded data corresponding to the respective 

ing relevant data relating to individual data to be recorded or objects is controlled using the information shown in the 

transmitted, wherein the relevant data includes an object object table stored in the table storage means. Therefore, it 

table on which N pieces of object data (N-positive integer) is possible to realize a decoding apparatus of simple struc- 

constituting the individual data and having a hierarchical ture that can perform edition, replacement or retrieval of 

structure are correlated with N pieces of coded data obtained objects (video and audio) constituting one scene in short 

by compressively coding the respective object data. time with high controllability. 

Therefore, extraction, selection or retrieval of specific object According to a twenty-third aspect of the present 

data from the individual data can be canied out easily and invention, there is provided an object data processing appa- 

speedily with reference to the object table. ^^^^^ outputting multiplexed data including N pieces of 

According to a nineteenth aspect of the present invention, coded data (N=positive integer) obtained by compressively 

there is provided a data storage medium containing relevant coding scene data corresponding to one scene, for each of N 

data corresponding to one scene, wherein the relevant data pieces of objects constituting the scene. This apparatus 

includes an object table on which N pieces of coded data includes hierarchical information extraction means for 
(N-positive integer) obtained by compressively coding 45 extracting hierarchical information showing the hierarchical 

scene data corresponding to one scene for each of N pieces relationship of the respective objects constituting the scene, 

of objects constituting the scene are correlated with the according to information showing the correlation of the 

respective objects. Therefore, on the decoder side, respective coded data included in the multiplexed data; and 

extraction, selection or retrieval of a specific object from table creation means for creating, according to the hierar- 
plural objects (video and audio) constituting one scene can 50 chical information, an object table on which the respective 

be performed easily and speedily with reference to the object objects are correlated with coded data corresponding to the 

table. respective objects. This apparatus outputs the multiplexed 

According to a twentieth aspect of the present invention. data to which the object table corresponding to the multi- 
there is provided an object data processing apparatus for plexed data is added. Therefore, it is not necessary to create 
outputting N pieces of coded data (N=posilive integer) 55 an object table on the decoder side, whereby edition, 
obtained by compressively coding N pieces of object data replacement or retrieval of objects (video and audio) con- 
which constitute individual data to be recorded or transmit- stituting one scene can be performed by a simple structure, 
ted and have a hierarchical structure, for each object data. in short time with high controllability. 
This apparatus includes hierarchical information extraction According to a twenty-fourth aspect of the present 
means for extracting hierarchical information showing the 60 invention, there is provided an object data processing appa- 
hierarchical relationship of the N pieces of objea data, ratus for decoding data output from the object data process- 
according to the coded data; and table creation means for ing apparatus according to the twenty-third aspect. This 
creating, according to the hierarchical information, an object apparatus includes data separation means for separating the 
table on which the respective object data are correlated with object table from the output data; and table storage means 
coded data corresponding to the respeaive object data. This 65 for storing the separated object table. In this apparatus, 
apparatus outputs the N pieces of coded data to which the decoding of the coded data corresponding to the respective 
object table corresponding to these coded data is added. objects is controlled using the information shown in the 
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object table stored in the table storage means. Therefore, it object table showing the hierarchical relationship of the 

is possible to realize a decoding apparatus of simple struc- plural packets constituting the multiplexed data. Therefore, 

lure that can perform edition, replacement or retrieval of extraction, selection or retrieval of a specific object from 

objects (video and audio) constituting one scene in short plural objects (video and audio) constituting one scene on 

time with high controllability. 5 the basis of the multiplexed data can be carried out easily 

According to a twenty-fifth aspect of the present and speedily with reference to the object table, 

invention, there is provided a data structure for transmitting According to a twenty-ninth aspect or the present 

N pieces of coded data (N-positive integer) obtained by invention, there is provided an object data recording appa- 

compressively coding N pieces of object data which con- ratus having a data storage for storing data, and recording, 

stitute individual data to be recorded or transmitted and have ^ in the storage, multiplexed data which includes N pieces of 

a hierarchical structure, for each object data. In this data coded data (N=positive integer) and is partitioned into plural 

structure, a data group comprising the N pieces of coded packets each packet having a prescribed coded quantity, 

data includes an object table on which the respective object which coded data are obtained by compressively coding N 

data are correlated with coded data corresponding to the Pieces of object data which constitute individual data to be 

respective object data. Therefore, extraction, selection or recorded or transrnUted and have a hierarchical stm^ 

retrieval of coded data corresponding to a specific object can ^^^^ object data. This apparatus mcludes hierarchical infor- 

be carried out easily and speedily with reference to the '^^jl^'^ extraction means for extracting hierarchical infor- 

.. .. ^ ^ ^ mation showmg the hierarchical relationship of the N pieces 

o jec a e. of object data, according to information showing the corre- 

According to a twenty-sixth aspect of the present 1^^^^^ ^^e respective coded data and included in the 

invention, there is provided a data structure for transmittmg 20 multiplexed data; and table creation means for creating, 

N pieces of coded data (N=positive integer) obtained by according to the hierarchical information, an object table 

compressively coding scene data corresponding to one showing the hierarchical relationship of the plural packets 

scene, for each of N pieces of objects constituting the scene. constituting the multiplexed data. This apparatus records the 

In this data structure, a data group comprising the N pieces multiplexed data and the object table corresponding to the 

of coded data includes an object table on which the respec- 25 multiplexed data in the data storage. Therefore, extraction, 

tive objects are correlated with coded data corresponding to selection or retrieval of coded data corresponding to a 

the respective objects. Therefore, on the decoder side, specific object can be carried out easily and speedily with 

extraction, selection or retrieval of a specific objected from reference to the object table. 

plural objects (video and audio) constituting one scene can According to a thirtieth aspect of the present invention, 

be carried out easily and speedily with reference to the 30 there is provided an object data recording apparatus having 

object table. a data storage for storing data, and recording, in the data 

According to a twenty-seventh aspect of the present storage, multiplexed data which includes N pieces of coded 
invention, there is provided an object data processing appa- data (N — positive integer) and is partitioned into plural 
ratus for processing multiplexed data including N pieces of packets each having a prescribed coded quantity, which 
coded data (N-positive integer) and being partitioned into 35 coded data are obtained by compressively coding scene data 
plural packets each having a prescribed code quantity, which constituting one scene, for each of N pieces of objects 
coded data are obtained by compressively coding N pieces constituting the scene. This apparatus includes hierarchical 
of object data which constitute individual data to be information extraction means for extracting hierarchical 
recorded or transmitted and have a hierarchical structure, for information showing the hierarchical relationship of the 
each object data. This apparatus includes hierarchical infor- 40 respective objects constituting the scene, according to infor- 
mation extraction means for extracting hierarchical infor- mation showing the correlation of the respective coded data 
mation showing the hierarchical relationship of the N pieces and included in the multiplexed data; and table creation 
of object data, according to information showing the corre- means for creating, according to the hierarchical 
lation of the respective coded data and included in the information, an object table showing the hierarchical re la- 
multiplexed data; and table creation means for creating, 45 tionship of the plural packets constituting the multiplexed 
according to the hierarchical information, an object table data. This apparatus records the multiplexed data and the 
showing the hierarchical relationship of the plural packets object table corresponding to the multiplexed data in the 
constituting the multiplexed data. Therefore, extraction, data storage. Therefore, on the decoder side, extraction, 
selection or retrieval of coded data corresponding to a selection or retrieval of a specific object from plural objects 
specific object on the basis of the multiplexed data can be 50 (video and adagio) constituting one scene can be carried out 
carried out easily and speedily with reference to the object easily and speedily with reference to the object table, 
table, and this enables the user to edit or replace the object ^^^^^ DESCRIFnON OF TOE DRAWINGS 
data in short time with high controllabihty. 

According to a twenty-eighth aspect of the present F^G. 1 is a block diagram for explaining an object data 

invention, there is provided an object data processing appa- 55 decoding apparatus as an object data processing apparatus 

ratus for processing multiplexed data including N pieces of accordmg to a first embodiment of the present mvention. 

coded data (N-positive integer) and being partitioned into FIG. 2 is a diagram showing an object table created by the 

plural packets each having a prescribed code quantity, which object data decoding apparatus according to the first 

coded data are obtained by compressively coding scene data embodiment. 

corresponding to one scene, for each of N pieces of objects 60 FIG, 3 is a flowchart for explaining process steps by a 

constituting the scene. This apparatus includes hierarchical CPU in the object data decoding apparatus according to the 

information extraction means for extracting hierarchical first embodiment. 

information showing the hierarchical relationship of the FIG. 4 is a diagram for explaining another object table 

respective objects constituting the scene, according to infor- created in the object data decoding apparatus according to 

mation showing correlation of the respective coded data 65 the first embodiment, illustrating a table corresponding to 

included in the multiplexed data; and table creation means upper-layer objects and a table corresponding to lower-layer 

for creating, according to the hierarchical information, an objects. 
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FIG. 5 is a block diagram for explaining an object data data obtained by the decoding to the display unit 14. The 

selecting apparatus as an object data processing apparatus coded data is identical to the multiplexed bit stream MEg 

according to a second embodiment of the present invention. shown in FIG. 19 in which coded data obtained by object 

FIG. 6 is a flowchart for explaining process steps by a coding of scene data corresponding to a single scene corn- 
CPU in the object data selecting apparatus according to the 5 prising plural objects are multiplexed with auxiliary data, 
second embodiment Th^ single scene corresponds to an image of each frame 

FIG. 7 is a block diagram for explaining an object data constituting a motion picture. The object data decoding 

recording apparatus according to a third embodiment of the f P^^^!^^ l«f successively decodes coded data of each 

present kivention ^^^^ ^"P^^ thereto, and successively outputs regeneration 

^ . „ ' r 1.. 1 10 data corresponding to each frame. 

FIG. 8 IS a flowchart for explainmg process steps by a ^^^^ specifically, the object data decoding apparatus 101 

CPU in the object data recording apparatus according to the i^^iud^^ demultiplexer 11 and an AV (audio/video) decoder 

third embodunent. ^2. The demultiplexer U selects and extracts a composition 

FIG. 9 is a block diagram for explaining an object data stream and a stream association table which arc auxiliary 

oulputling apparatus as an object data processing apparatus 15 data Dsub included in the multiplexed bit stream Meg, and 

according to a fourth embodiment of the present invention. outputs coded data Eg corresponding to the respective 

FIG. 10 is a flowchart for explaining process steps by a objects in the multiplexed bit stream MEg, in units of the 

CPU in the object data outputting apparatus according to the respective objects, according to a first control signal Contl. 

fourth embodiment. The AV decoder 12 decodes the coded data Eg correspond- 

FIG. 11 is a diagram for explaining an object data 20 ing to each object according to a second control signal 

decoding apparatus based on MPEG4 as an object data Cont2, and outputs regeneration data Rg corresponding to 

processing apparatus according to a fifth embodiment of the each scene. Further, the decoding apparatus 101 includes a 

invention, illustrating the outline of a data transmission CPU (central processing unit) 13. The CPU 13 decides a 

system based on MPEG4. logical channel of a packet containing the coded data Eg of 

HG. 12 is a block diagram for explaining an object data ^5 each object according to the stream association table and 

decoding apparatus as an object data processing apparatus ^"PP''^* first contro signal Contl to the demultiplexer 

according to the fifth embodiment of the invention. \\ accordmg to the result of the decision. Further the CPU 

T-w^o -.-^^ X , ^-^rix . ■ f r 13 supplies information relating to location of each object in 

FIGS. 13(«) and 13(6) are schematic diagrams for information relating to display start time of 

explaimng an object coding method correspondmg to the ^^^^ ^^^^ control signal CoDt2, to the AV 

data transmission system shown m FIG, 11. j j l • c j • j 

^ decoder 12, on the basis of a scene descnption according to 

FIG. U(a) is a diagram showing a scene description and ^^le composition stream. 

FIG. 14(^) is a diagram showing object descriptors, In this first embodiment, the CPU 13 creates an object 

respectively, used in the data transmission system. table showing the correspondences between the respective 

FIGS. 15(fl) and 15(b) are diagrams showing an object 35 objects constituting one scene and coded data Eg of the 

table obtained from the scene description shown in FIG. respective objects in the multiplexed bit stream MEg, on the 

14(fl) and the object descriptors shown in FIG. 14(b), basis of the composition stream and the stream association 

FIG. 16 is a diagram showing a flow of process steps by table, and stores this table in a data storage inside the CPU 

a CPU in the object data decoding apparatus according to the 13- 

fifth embodiment. 40 FIG. 2 is a diagram for explaimng an object table Tl 

FIGS, 17(a)-17(c) are diagrams for explaining a data corresponding to the scene 120 shown in FIG. 18. 

storage medium according to the present invention, wherein ^" ^^i^^^ ^able Tl, various kinds of information are 

r^n \ u *u 4 * V a j- i entered, correlated with each obiect iodcx (obiect id) which 

FIG. l7(a) shows the structure of a floppy disk, • \ c ■ t. t.- ^ - - r 

-i-/. ^ . . ^ n i- . . . , IS an index fur identifying each obi ectbeme a constituent of 

FIG. 11(b) shows the stnicture of a floppy disk body, and ^^^^ ^^^^ ^^.^^^ ^^^^^ ^^.^^^ 

FIG. 17(c) shows a computer system using the floppy disk index, 

as a storage medium. jq specific, object indices are uniquely given to the 

FIG. 18 is a schematic diagram for explaining an object respective object descriptors in the scene description shown 

coding method according to the prior art. in FIG. 20, and entered to the object table Tl in the order of 

FIG. 19 is a diagram showing a data stmcture of a bit 50 the object id values Oid. 

stream obtained by multiplexing data coded by the prior art In FIG. 2, object indices Oid=l to Oid=5 are given to the 

object coding method and auxiliary data. descriptors 141 to 145 of the objects 121 to 125 shown in 

FIG. 20 is a diagram showing a scene description accord- FIG. 20, respectively, and object indices Oid=61 and Oid=62 

ing to a composition stream included in the bit stream shown gi^^" to the descriptors 148 and 149 of the objects 128 

in FIG. 17 as auxiliary data. 55 and 129 shown in FIG. 20, respectively. Further, an object 

FIG. 21 is a diagram showing a stream association table ^"^^^ ^'^'^ ^ ^Wen to the descriptor 147 of the object 127 

included in the bit stream shown in FIG. 17 as auxiliary data, ^° 

On the object table Tl, correlated with each object index 

DETAILED DESCRIPTION OF THE (id), the following components are entered: a logical channel 

PREFERRED EMBODIMENTS 60 (LC) corresponding to the object; the stream type (i.e., 

[Embodiment 1] whether the stream is video or audio); a stream index 

FIG. 1 is a block diagram fllustrating an object data corresponding to the object; indices of upper and lower 

decoding apparatus as an object data processing apparatus objects corresponding to the object; logical channels of the 

according to a first embodiment of the present invention. upper and lower objects; the index of an object which shares 

In FIG. 1, an object data decoding apparatus 101 receives 65 its logical channel with the object (common object id in FIG. 

coded data corresponding to a single image (scene), per- 2), and the priority order of the respective objects. In FIG. 

forms decoding of the coded data, and outputs regeneration 2, OLC is a specific value of logical channel LC. 
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An upper object of each object is an object which belongs At this time, in the CPU 13, information relating to each 

to an upper layer in the hierarchical structure than a layer to object's display position and display start time is extracted 

which each object belongs. A lower object of each object is from the scene description according to the composition 

an object which belongs to a lower layer than a layer to stream, and the extracted information is output to the AV 

which each object belongs. 5 decoder 12 as a second control signal Cont2. 

To be specific, the uppermost object, i.e., an upper object the AV decoder 12, a stream (a series of coded data Eg) 

of the objects belonging to the first layer LI in FIG, 18, is corresponding to each object output from the demultiplexer 

the object itself. So, the uppermost object has its object id subjected to decoding. Decoded data corresponding to 

value Oid-1 and its logical channel value OLC-3 for its respective objects are composted according to the second 

upper object id value Oid and its ^PP^r object logical ^^^^^j ^. ^ ^^^^ ^^^^ ^3 information 

channel value OLC. Further, the upper objects of the respec- . u'*j*i\ j .- j^n 

live objects belonging to the second layer U and the third '«'*"".8 '° ""Jf ' display) and regeneration data Rg corre- 

layer L3 in FIG. 18 are the objects belonging to the first layer ^P°°'*'"8 to the scene 120 composed of plural objects is 

LI and composed of the objects of the second and third output. - . -i . 

layers L2 and L3, Further, objects having no lower objects decoding operation menUoned above is similar to that 

(lowermost objects) have lower object id values Oid-0. In 15 of the conventional object data decoding apparatus, 

this first embodiment, a specific objects being the uppermost The object data decoding apparatus 101 according to this 

object is shown by giving its object id as its upper object id, first embodiment is characterized by that the object table Tl 

and a specific object's being the lowermost object is shown shown in FIG. 2 is created by the CPU 13. 

by giving Oid=0 as its lower object id. However, a special Hereinafter, a description is given of the object table 

digit or symbol may be used for describing that a specific 20 creation process. FIG. 3 is a flowchart showing the algo- 

object is the uppermost or lowermost object. rithm for creating the object table Tl by the CPU 13. 

From the object table Tl so constructed, it can be seen that Initially, in step SI, the composition stream is read into 

the object having Oid=2 as its object id is composed of four the data storage of the CPU 13. In step S2, the stream 

objects because "Oid=4, 5, 6, 8" is described as its lower association table is read into the data storage of the CPU 13. 

object id. 25 In step S3, descriptors of the respective objects on the 

Further, in FIG. 2, for the object 128 (129) which is a composition table are loaded into a processor of the CPU 13 

component of the object 126 corresponding to a node having wherein an objea id value Oid is given to each descriptor, 

no stream (in the description of FIG. 20, Node(2) 146), its whereby each object can be identified by the object id. 

object id is given as follows. That is, a digit showing that this In step S4, it is decided whether or not the object of which 

is a node common to some objects is given as the upper 30 descriptor has been loaded corresponds to a node and has no 

column of its Oid, and a digit showing its actual object id stream. When it corresponds to a node and has no stream, in 

value is given as the lower column of the Oid. step Sll, the layer of object of which descriptor is to be 

Accordingly, from the object table Tl, it is found that the loaded is lowered by one, followed by step S3 wherein the 

object 126 having its object id value Oid=6 is merely a node descriptors of the lower objects being components of the 

and there exists no coded data corresponding to this node, 35 object corresponding to the node arc loaded into the CPU 13. 

and the stream of this object 126 is composed of the streams When it is decided in step S4 that the object of which 

of the objects 128 and 129 having object ids Oid=61 and descriptor has been loaded is not one corresponding to a 

Oid-62, i.e., the streams of Sid-6 and Sid -7, respectively. node and having no stream, in step S6, the object id value 

Further, the object 122 having Oid-2 as its object id Oid is entered as a component of the object table, 
corresponds to a node, like the object 126 having Oid-6 as 40 Thereafter, in step S7, stream association table is inter- 
its object id. However, since this object 122 is definitely preted and, according to the result of the interpretation, 
described on the object table Tl, there exists a stream various kinds of table components corresponding to each 
corresponding to this object 122, i.e., the stream of Sid=2. object are entered in the object table. The main table 

Information included in a stream corresponding to a node components are as foUows: the logical channels (LC) cor- 

is information common to all objects belonging to the node, 45 responding to the respective objects, the priority order of the 

for example, composition information peculiar to the respective objects, the stream indices (id) corresponding to 

objects, such as common system clock, display start timing, the respective objects, and the stream type (i.e., Video or 

decoding start time, and display position, and copyright Audio). Besides, the following table components are also 

information. entered: the indices of the upper and lower objects corre - 

A description is now given of the operation of the object 50 spending to the respective objects, the logical channels of 

data decoding apparatus 101. the upper and lower objects, and the indices of objects which 

When the multiplexed bit stream MEg shown in FIG. 19 share their logical channels with other objects, 

is input to the object data decoding apparatus 101, the In step S8, it is decided whether entry of table components 

composition stream and the stream association table relating to objects that belong to the same node as an object 

(auxiliary data Dsub) are extracted from the multiplexed bit 55 currently being processed by the CPU 13 has been com- 

stream MEg and supplied to the CPU 13 by the demulti- pleted or not. When it has not been completed, the control 

plexer 11. of the CPU 13 returns to step S3, followed by steps S4 to S8. 

In the CPU 13, the correspondences between the respec- On the other hand, when it is decided in step S8 that entry 

tive objects constituting the scene 120 shown in FIG. 19 and of table components has been completed with respect to all 

their logical channels LC are recognized and, according to 60 the objects of the node to which the object currently being 

the recognition, a first control signal Contl is output to the processed belongs, the CPU control proceeds to step S9 

demultiplexer 11. wherein it is decided whether or not the object currently 

In the demultiplexer 11., according to the first control being processed is the uppermost-layer object in the hierar- 

signal Contl, streams corresponding to the respective chy. 

objects, which are allocated to plural packets in the multi- 65 When the object currently being processed is not the 

plexed bit stream, are output to the AV decoder 12 in object uppermost- layer object, a process of raising the object layer 

units. by one is carried out in seep S12, followed by the decision 
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in step S9. On the other hand, when it is decided in step S9 In this first embodiment, since the object table Tl contains 

that the object currently being processed is the uppermost- the index of object that shares its logical channel with 

layer object, the CUP control proceeds to step SIO wherein another object, this index can be used for deciding whether 

it is decided whether entry of table components has been coded data of the object can be deleted or not, whereby the 

completed or not with respect to all the objects constituting 5 above-mentioned problem is avoided, 

one scene. described above, in this first embodiment of the 

When it is decided that entry of table components of all invention, on the basis of the multiplexed bit stream includ- 

the objects has not been completed yet, the CPU control ■ ^^^^ data Eg corresponding to plural objects, the object 

returns to step S3 followed by steps S4 to S9 Sll and S12. ^^^^ ^^e correspondences between the respective 

On the other hand, when the dcc^ion m step 10 is that ent^^ ^^.^^ ^^^^^ ^^^^ ^^^^^^^ ^^^^^^ 

of table components 01 all the objects has been completed, t.- * * ui * *■ 1 » • 1 * 

the object table creation process by the CPU 13 is ended. ^^e object table, extraction, selection, or retneval of a 

In the object data decoding apparatus 101 according to ^P^^^^^ ^ ^^"^f out. Therefore, as compared with 

this first embodiment, the object table so created is stored in *he case where retneval or the hke of a specific object is 

the data storage of the CPU 13. The stored object table will earned out by interpretmg information about the object 

be updated at every updating of the composition stream and ^5 included m the multiplexed bit stream each time, the pro- 

the stream association table so that it can correspond to the cessing quantity required for retrieval or the like is reduced, 

updated information. Accordingly, the object table is resulting in high-speed processing. 

updated only when any of the objects constituting one scene Further, on the object table Tl, since the correlation of 

is changed. plural objects constituting one scene and having a hierar- 

When the multiplexed bit stream includes a flag showing 20 chical structure is described clearly, even when plural 

that the updated composition stream and stream association objects are included in one object, replacement and edition 

table are sent, the object table may be updated only when the of the objects are facilitated. 

flag is newly sent. Although in this first embodiment the objects that share a 

When the object data decoding apparatus 101 performs logical channel are shown by their object ids, flags such as 

decoding to a specific object according to a request from the 25 "1" and "0" may be used to show only whether or not an 

user, a logical channel LC corresponding to the specific object shares its logical channel with another object is 

object is specified on the basis of the object table stored in required. In this case, the object table is simplified, 

the data storage of the CPU 13, and only a packet having this Further, the object table is not restricted to that shown in 

logical channel LC is extracted from the multiplexed bit FIG. 2. Hereinafter, a description is given of an object data 

stream for decoding. 30 decoding apparatus which creates an object table different 

For example, when only the object 122 (Oid='2) corre- from the object table shown in FIG. 2, according to a 

spending to a node is subjected to decoding, in the multi- modification of the first embodiment, 

plexed bit stream of the data structure according to the prior FIG. 4 is a diagram for explaining an object table created 

art, it is necessary to interpret the composition stream and by the decoding apparatus according to the modification, 

the stream association table by the CPU and specify the 35 wherein a table corresponding to upper-layer objects and a 

logical channel LC corresponding to the object 122. In this table corresponding to lower-layer objects are illustrated, 

first embodiment, however, since the object table shown in The object table T2 shown in FIG. 4 is different from the 

FIG. 2 is included in the multiplexed bit stream MEg, the object table Tl shown in FIG. 2 in that the table T2 does not 

logical channel LC corresponding to the object 122 (i.e., have logical channels (LC) of the respective objects, and the 

OLC-6-9) can be specified in a moment, resulting in 40 table T2 is divided into two parts, i.e., an upper-layer table 

high-speed retrieval. T2a and a lower-layer table T2b. 

Further, when the throughput of the decoding apparatus is The process of creating the object table T2 is different 

low and it cannot decode all the objects, it is considered to from the process flow shown in FIG. 3 only in that the 

decode only objects having high priorities. In this first logical channels (LC) are not entered as table components, 

embodiment, since the object table contains the priority 45 So, the object table T2 can be created according to a process 

order of the respective objects, it is easy to specify the flow similar to the process flow shown in FIG. 3. 

logical channels of high-priority objects. By the way, as mentioned above, the bit stream is not 

Furthermore, since the object table contains the indices of always one in which coded data corresponding to the 

objects that share a logical channel with other objects, the respective objects are multiplexed. In a bit stream in which 

following effect is expected. 50 coded data are not multiplexed, no stream association table 

That is, in the bit stream according to this embodiment, is included. So, no logical channel LC is obtained from this 

the objects 124 and 125 having Oid=4 and Oid»5 as their bit stream. 

object ids (see FIG. 18) share coded data of the same logical In such a bit stream, the logical channels ii the multi- 
channel LC (0LC=6), So, although these objects have plexed bit stream correspond to streams, so that coded data 
different object id values and different stream id values, 55 of each object is specified using the stream id instead of the 
these objects have the same value (OLC-6) Or correspond- logical channel LC. 

ing logical channel LC. Therefore, on the object table T2 shown in FIG. 4, like the 
When coded data of a specific object is deleted from the object table Tl shown in FIG. 2, an object index (id) for 
multiplexed bit stream, according to the object id of an identifying each object is entered to make the relationship 
object relating to the specific object, a logical channel LC 60 between the object and the stream clear, 
corresponding to the relevant object is decided and, Further, on the object table T2 shown in FIG. 4, in order 
thereafter, coded data of the specific object is extracted from to make the object hierarchy clear, "H" is adopted as a code 
the multiplexed bit stream. However, when the specific showing hierarchical information and described in the sec- 
object shares its logical channel with the relevant object, if tion of the kind of stream. With respect to the object having 
coded data of the specific object is deleted, coded data 65 "H" in this section, the object table T2b (lower-layer table) 
corresponding to the relevant object is gone, whereby corresponding to lower-layer objects included in this object 
decoding of the relevant object cannot be carried out. is created. 
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As described above, since the object table has a hierar- and the object selector 62, and retains the multiplexed bit 

chical structure, even when the hierarchy of objects consti- stream for a prescribed period of time, 

tuting one scene increases, the size of the object table for Further, the selecting apparatus 102 includes a CPU 63. 

each layer ijla or TZb), which is a component of the object The CPU 63 creates the object table Tl shown in FIG. 2 on 

table (T2), does not increase. Therefore, when perfonning 5 ^he basis of the composition stream and the stream associa- 

edition or replacement to the upper-layer objects, only the ^^ble, and outputs a signal for selecting coded data of a 

upper-layer object table (T2a) of which size is reduced is specified object (control signal Cont) toward the object 

retrieved, whereby detection of objects is facilitated, ^^^^^'^^ according to an object specifying signal lu 

Although in the first embodiment and its modification, generated as a result of user s operation. Smce a)ded data of 

object tables are obtained from the multiplexed bit stream lo '^"^ ^P^?f^^ ^^^^ ^ ^""'"'^'^^ the multiplexed bit 

u • T-T/- 'y J .u 1." 1 * u stream MEg, the contents described in the composition 

sho^ in FIG. 3 and Ihe non-multiplexed bit stream shown ^(^^^^ 3,^^^^ association table change. So, the CPU 

in FIG. 4, respectively, object tables are not restricted „ j^^^j,^^ ^^^^-^^^ ^^^^ ^gj3,j„g ,^ composition 

ttiereto. stream and the stream association table so that the stream 

For example, m order to make the object table shown in ^^^d the table correspond to the extracted object, and outputs 

FIG. 2 compact, from the table components shown in FIG. 15 the data to the object selector 62. the object selector 62 adds 

2, the upper object id, the upper object LC, the priority order, the composition stream and the stream association table, 

the kind of stream, and the common object id may be which have been rewritten by the CPU 13, to the coded data 

deleted. of the extracted object when outputting the coded data. 

To the contrary, although the size of the object table is A description is given of the operation of the object data 

somewhat increased, in order to simplify the operation such 20 selecting apparatus 102 using a flowchart shown in FIG. 6. 

as edition or replacement of objects, the logical channel of Initially, in step S71, the object table Tl is created 

the composition stream itself and the logical channel of the according to the multiplexed bit stream MEg input to the 

stream association table itself may be described on the object data selecting apparatus 102. 

object table, or header information of streams corresponding To be specific, the multiplexed bit stream is interpreted by 

to video and audio objects may be added to the table. 25 the multiplexed stream interpreter 61, and the composition 

As described above, the object table created by the object stream and the stream association table which are auxiliary 

data decoding apparatus according to the present invention data Dsub included in the multiplexed bit stream MEg are 

may have any structure as long as, on the table, the respec- detected and supplied to the CPU 63. In the CPU 63, the 

tive objects are correlated with the stream indices or logical object table Tl is created according to the auxiliary data 

channels of the objects. 30 Dsub and stored in the data storage. The process of creating 

Furthermore, in the object coding method, there is a case the object table Tl is identical to that already described for 

where the multiplexing relationship is expressed by only the the first embodiment. 

stream association table in the bit stream without including When a signal lu specifying an object is input by the user 

the composition stream in the bit stream, in order to simplify or the like (step 372), the CPU 63 retrieves the object table 

the structure of the decoding apparatus. 35 according to the object specifying signal lu, specifies the 

In this case, although the object-to-stream correspondence logical channel LC of the specified object, and sends the 

is not uniquely defined, an object table excluding some of logical channel LC to the object selector 62 as the control 

table components of the object table Tl shown in FIG. 2, signal cont (step S73). 

such as the object id, the stream id, the kind of stream, the Subsequently, the CPU 63 rewrites the composition 

priority order, and the common object id, may be created 40 stream (step S74) and rewrites the stream association table 

from the stream association table. In this case, the hierar- (step S75) so that the stream and the table correspond to 

chical relationship of the logical channels corresponding to coded data of the specified object. 

coded data of the respective objects is clearly defined. To rewrite the composition stream and the stream asso- 

Further, an object table excluding some of table components ciation table is necessary because the correlation of objects 

of the object table T2 shown in FIG. 4, such as the object id, 45 included in the multiplexed bit stream changes between the 

the stream id, the kind of stream, and the priority order, may input multiplexed bit stream and the multiplexed bit stream 

be created from the stream association table. corresponding to the extracted object. While the rewriting is 

[Embodiment 2] carried out, the multiplexed bit stream output from the 

FIG. 5 is a block diagram for explaining an object data multiplexed stream interpreter 61 is stored in the buffer 64. 

selecting apparatus as an object data processing apparatus 50 When coded data corresponding to each object included 

according to a second embodiment of the present invention. in the multiplexed hit stream is input through the buffer 64 

In FIG. 5, an object data selecting apparatus 102 accord- to the object selector 62 (step S76), the object selector 62 

ing to this second embodiment selects and extracts coded decides whether the input coded data corresponds to the 

data Sg of a specific object from a multiplexed bit stream logical channel LC of the specified object or not, according 

MEg, according to an instruction signal lu corresponding to 55 to the control signal Cont corresponding to the object 

user's instruction. The multiplexed bit stream MEg is iden- specifying signal from the CPU 63 (step S77). 

tical to that shown in FIG. 19 wherein coded data of plural As the result of this decision, when the input coded data 

objects are multiplexed in units of the objects. does not correspond to the specified object, the next coded 

The object data selecting apparatus 102 includes a mul- data is input to the object selector 62 (step S76). On the other 

tiplexed stream interpreter 61, an object selector 62, and a 60 hand, when the input coded corresponds to the specified 

buffer 64. The multiplexed stream interpreter 61 detects a object, the coded data is output as coded data corresponding 

composition stream and a stream association table which are to the specified object (step S78). The object selector 62 

auxiliary data Dsub included in the multiplexed bit stream outputs the coded data of the selected object together with 

MEg. The object selector 62 selects and extracts coded data the rewritten composition stream and stream association 

corresponding to a specific object from the multiplexed bit 65 table. 

stream MEg according to a control signal Cont. The buffer Thereafter, the object selector 62 decides whether or not 

64 is disposed between the multiplexed stream interpreter 61 the output coded data is the last coded data of the specified 
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object. As the result of this decision, when the output coded 
data is not the last one, above -nientioned Steps S76-S79 are 
repeated. On the other hand, when the output coded data is 
the last one, it is decided whether output of coded data of all 
the specified objects is completed or not (step 380). When it 5 
is not completed yet, above-mentioned steps S76-S80 are 
repeated. On the other hand, when it is completed, the 
process of selecting coded data of specified objects is ended. 

When the object selector 62 outputs coded data of a 
specified object as described above, the transfer rate of the 
multiplexed bit stream output from the object data selecting 
apparatus 102 is lowered as compared with the transfer rate 
of the multiplexed bit stream input to this apparatus. So, the 
object selector 62 can change the transfer rate as desired. 
However, the transfer rate may be changed by independent 
means located on the output side of the selector 62. 

As described above, according to the second embodiment 
of the invention, on the basis of a bit stream in which coded 
data Eg corresponding to plural objects are multiplexed, an 
object table on which the respective objects are correlated 
with the coded data thereof is created in advance. Using the 20 
object table, coded data corresponding to a specific object is 
extracted from the bit stream. Therefore, it is possible to 
extract or delete coded data of a specific object from the 
multiplexed bit stream at high speed in the middle of a 
transmission path or the like. 25 

Although in this second embodiment a specific object 
extracted is transmitted, a specific object extracted may be 
deleted to transmit the rest of the stream. 
[Embodiment 3] 

FIG. 7 is a block diagram for explaining an object data 30 
recording apparatus according to a second embodiment of 
the present invention. 

In FIG. 7, an object data recording apparatus 103 includes 
a data storage 84, and an object data selecting unit 8 that 
selects and extracts data stored in the data storage 84. The 35 
recording apparatus 103 records the muhiplexed bit stream 
MEg in the data storage 84, and retrieves or outputs coded 
data of a specified object from the stream stored in the data 
storage 84 according to user's instruction or the like. 

The object data selecting unit 8 includes a multiplexed 40 
stream interpreter 81 and an object selector 82. The multi- 
plexed stream interpreter 81 detects a composition stream 
and a stream association table which are auxiliary data Dsub 
included in the multiplexed bit stream MEg, The object 
selector 82 selects and extracts coded data corresponding to 45 
a specific object from the multiplexed bit stream MEg 
according to a control signal Cont. 

Further, the object data selecting unit 8 includes a CPU 
83. The CPU 83 creates the object table Tl shown in FIG. 
2 on the basis of the composition stream and the stream 50 
association table, and records the object cable in a prescribed 
region of the data storage 84, for example, a region where 
management information showing the contents of the stor- 
age 84 is recorded, or a region where object tables are 
managed collectively. The object table may be recorded in 55 
the same region as where the multiplexed bit stream MEg is 
stored so that the table is positioned at the head of the stream 
MEg. 

Further, the CPU 83 outputs a signal for selecting coded 
data of a specified object (control signal Cont) toward the 60 
object selector 82 according to an object specifying signal lu 
generated as a result of user's operation. Furthermore, the 
CPU 83 rewrites the auxiliary data relating to the compo- 
sition stream and the stream association table so that the 
stream and the table correspond to the selected object, and 65 
adds the rewritten data to the coded data of the selected 
object when the coded date is output. 



A description is given of the operation of the object data 
recording apparatus 103. 

FIG. 8 is a flowchart showing process steps of creating the 
object table. 

When the multiplexed bit stream MEg is input to the 
object data recording apparatus 103 and stored in the data 
storage 84 (step S91), an object table corresponding to the 
multiplexed bit stream is created in the object data selecting 
unit 8 controlled by the CPU 83. 

More specifically, the multiplexed bit stream is input to 
the multiplexed stream interpreter 81 (step S92), and the 
composition stream and the stream association table which 
are auxiliary data Dsub included in the multiplexed bit 
stream MEg are detected by the interpreter 81 and supplied 
to the CPU 83, In the CPU 83, an object table (refer to FIG. 
2) is created according to the auxiliary data Dsub (step S93). 
The object table created by the CPU 83 is stored in the data 
storage 84 (step 894). Thereafter, it is decided whether an 
instruction to end the process of selecting object data is input 
or not. When there is no end instruction, above-mentioned 
steps S91~S95 are repeated. When the end instruction is 
input, the object data selecting process is ended. 

The object table is stored in a prescribed region of the data 
storage 84, for example, a region where management infor- 
mation showing the contents of the storage 84 is recorded, 
a region where object tables are managed collectively, or the 
same region as where the multiplexed bit stream MEg is 
stored (in this case, the table is stored at the head of the 
stream). 

When a signal specifying an object is input to the record- 
ing apparatus 103, in the CPU 83, the object table stored in 
the data storage 84 is retrieved, a logical channel LC 
corresponding to the specified object is specified, and the 
specified logical channel LC is output to the object selector 
82, 

In the object selector 82, on the basis of the logical 
channel LC from the CPU 83, coded data corresponding to 
the specified logical channel is selected from the multi- 
plexed bit stream, and the selected coded data Se is output. 
When the composition stream and the stream association 
table are changed due to the object selection, the CPU 83 
rewrites the stream and the table. The rewritten stream and 
table are input to the object selector 82 wherein the rewritten 
stream and table are added to the selected object to be 
output. 

As described above, according to the third embodiment of 
the present invention, in an apparatus for recording a mul- 
tiplexed bit stream including coded data of plural objects 
constituting a single image, an object table on which the 
respective objects are related with the coded data thereof is 
created on the basis of the multiplexed bit stream, and the 
coded data and the object table corresponding to the coded 
data are recorded. Therefore, the recorded multiplexed bit 
stream is collectively managed by the object table, whereby 
the process of retrieving and outputting a desired object 
from the recorded multiplexed bit stream is performed at 
high speed. 

In the object data recording apparatus 103, since the data 
storage 84 can serve as a buffer, no buffer is disposed 
between the multiplexed stream interpreter 81 and the object 
selector 82. However, a buffer as shown in FIG. 5 may be 
disposed between the interpreter 81 and the seleaor 82. 
[Embodiment 4] 

FIG. 9 is d block diagram illustrating an object data 
multiplex coding apparatus 104^ including an object data 
output unit 104 which is an object data processing apparatus 
according to a fourth embodiment of the present invention. 
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The object data multiplex coding apparatus 104^ com- 
prises an encoder 87 and the object data output unit 104. The 
encoder 87 generates coded data corresponding to plural 
objects constituting one scene by coding data of the respec- 
tive objects, multiplexes these coded data with a composi- 5 
tion stream and a stream association table which are auxil- 
iary data Dsub, and output the multiplexed data. The object 
data output unit, 104 adds an object table on which the 
respective objects are correlated with coded data of the 
objects to the multiplexed bit stream MEg output from the lo 
encoder 87, and outputs the multiplexed bit stream MEg 
with the object table. 

The object data output unit 104 includes a multiplexed 
stream interpreter 81 which detects the composition stream 
and the stream association table which are auxiliary data 15 
Dsub included in the multiplexed bit stream MEg according 
to a control signal Contl, and a buffer 85 which temporarily 
stores the multiplexed bit stream MEg that is input to the 
buffer 85 through the multiplexed stream interpreter 81. 
Further, the output unit 104 includes a CPU 83 which forms 20 
an object table Tl as shown in FIG. 2 on the basis of the 
composition stream and the stream association table, and a 
multiplexer 86 which adds the object table to the multi- 
plexed bit stream output from the buffer 85 according to a 
control signal Cont2. 25 

The operation of the object data output unit 104 will be 
described using a flowchart shown in FIG. 10. 

When the multiplexed bit stream generated by compres- 
sive multiplexing in the encoder 87 is input to the object data 
output Unit 104 (step Sill), the CPU 83 decides whether it 30 
is "start of scene" or "change of objects constituting one 
scene" (step S112). When the decision is neither of "start" 
and "change", the muhiplexer 86 is controlled by the control 
signal Contl so that the input multiplexed bit stream is 
output as it is. 35 

On the other hand, when it is "start of scene" or "change 
of objects constituting one scene", the multiplexed stream 
interpreter 81 is controlled by the control signal Contl so 
that the multiplexed bit stream MEg is input to the multi- 
plexed stream interpreter 81 and processed (step S113). In 40 
the CPU 83, the composition stream is detected from the 
multiplexed bit stream (step S114) and, subsequently, the 
stream association table is extracted from the multiplexed bit 
stream (step S115) and, further, an object table is created on 
the basis of the composition stream and the stream associa- 45 
tion table (step S116). 

Thereafter, in the CPU 83, at the time of scene start or 
scene change, the created object table is added at the head 
of the multiplexed bit stream output from the buffer 85 (step 
$117), and the multiplexed bit stream with the object table 50 
is output (step S118). 

Thereafter, it is decided whether an instruction to end the 
process of outputting the multiplexed bit stream from the 
encoder 87 is input or not (step S119). When there is the end 
instruction, the output process is ended. When there is no 55 
end instruction, above-mentioned steps Sill to S119 are 
repeated. 

As described above, according to the fourth embodiment 
of the invention, the object data output unit 104 receives a 
multiplexed bit stream obtained by multiplexing coded data 60 
of plural objects constituting one scene, adds an object table 
on which the respective objects are correlated with their 
coded data to the multiplexed bit stream, and outputs the 
multiplexed bit stream with the object table. Therefore, it is 
not necessary for an object data decoding apparatus receiv- 65 
ing the muUiplexed bit stream MEg with the object table to 
create an object table, whereby an object data decoding 



apparatus providing the same effects as the decoding appa- 
ratus according to the first embodiment can be realized with 
simplified structure. 

Employed as an object data decoding apparatus to which 
the multiplexed bit stream MEg and the object table are 
input may be either a decoding apparatus in which the object 
table and the coded data of the respective objects in the 
multiplexed bit stream MEg are stored in different storage 
regions or a decoding apparatus in which the object table and 
the coded data are stored in the same storage region. 

While in this fourth embodiment the object data output 
unit 104 outputs the multiplexed bit stream after adding the 
object table at the head of the stream, the structure of the 
output unit is not restricted thereto. For example, according 
to the application, the object table may be inserted in the 
middle of the multiplexed bit stream and, in this case, the 
capacity of the bluffer 85 can be decreased. 

Further, in this fourth embodiment, the encoder 87 Simply 
generates coded data corresponding to the respective 
objects, and the output unit 104 receives the multiplexed bit 
stream generated by the encoder and outputs the multiplexed 
bit stream after adding the object table to the stream. 
However, the structures of the encoder and the output unit 
are nut restricted thereto. For example, the encoder may 
create the object table simultaneously with formation of the 
composition stream and the stream association table, and 
add the object table to the multiplexed bit stream separately 
from the composition stream and the stream association 
table when outputting the multiplexed bit stream. Or, the 
object table may be output as a part of the composition 
stream. In this case, the multiple stream interpreter 81 in the 
output unit is dispensed with, and a conventional multi- 
plexer can be used in the output unit. 

Although in the second to fourth embodiments a multi- 
plexed bit stream is described as input coded data, these 
embodiments are not restricted thereto. 

For example, an input bit stream may be a bit stream in 
which coded data are partitioned in units of objects as 
described for the modification of the first embodiment. Also 
in this case, the same effects are obtained by creating the 
object table as shown in FIG. 4. 

Especially, even when the object data output unit 104 
according to the fourth embodiment is constructed so that it 
receives such a non-multiplexed bit stream and creates the 
object table shown in FIG. 4, employed as an object data 
decoding apparatus which receives the multiplexed bit 
stream MEg with the object table output from the output unit 
may be either a decoding apparatus in which the object table 
and the coded data of the respective objects in the multi- 
plexed bit stream MEg are stored in different storage regions 
or a decoding apparatus in which the object table and the 
coded data are stored in the same storage region. 

Furthermore, although in the first to fourth embodiments 
a multiplexed bit stream including coded data corresponding 
to video data and audio data is described, any multiplexed 
bit stream may be employed as long as it includes coded data 
of plural objects constituting individual information to be 
recorded or transmitted. For example, the respective 
embodiments may employ a multiplexed bit data including, 
as coded data of objects, only coded data of video data, 
audio data, or computer data, or a multiplexed bit data 
including coded data of other data. 

Furthermore, in the first to fourth embodiments, the object 
table is created on the basis of the composition table and the 
steam association table which are included in the multi- 
plexed bit stream MEg as auxiliary data. However, in an 
information transmission system corresponding to MPEG4 
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which is currently being standardized, the format of scene 
description is different from that according to the composi- 
tion table, and an object descriptor showing the correspon- 
dence between object id and stream id is employed in place 
of the stream association table showing the correspondence 
between stream id and logical channel. 

A description is now given of an object data transmission 
system according to MPEG4. 

FIG. 11 is a diagram illustrating the structure of the abject 
data transmission system 200. 

In this system 200, coded video data Ev and coded audio 
data Ea corresponding to objects constituting a single scene 
201 and system information Si as auxiliary data are multi- 
plexed by a multiplexer 202, and a multiplexed bit stream 
MEgl obtained as a result of the multiplication is transmit- 
ted through a transmission medium or stored in a storage 
medium. 

The multiplexed bit stream MEgl transmitted through the 
transmission medium or read from the storage medium is 
demultiplexed (divided) into the coded data Ev and Ea, and 
the system information Si by a demultiplexer 203. 

To be specific, the scene 201 is composed of a background 
object OBI (scenery), a sound object 0B4 attendant on the 
backgroimd object OBI, a foreground object 0B2 (person), 
and a voice object 0B3 attendant on the foreground object 
0B2. The coded video data Ev is divided into coded data 
Evl corresponding to the background object OBI and coded 
data Ev2 conesponding to the foreground object 0B2. The 
coded audio data Ea is divided into coded data Eal corre- 
sponding to the voice object 0B3 and coded data Ea2 
corresponding to the sound object 0B4. The system infor- 
mation Si as auxiliary data is divided into scene description 
information Sf and object descriptor OD. 

Receiving the respective data separated from the multi- 
plexed bit stream MEgl, a decoder 204 generates regenera- 
tion data Rg corresponding to the scene 201 according to 
these data. 

That is, in the scene description information Sf, the 
hierarchical structure of the objects 0B1--0B4 is described 
together with the relationship between the objects in each 
layer and their object indices. In the object descriptor OD, 
the relationship between the object indices and the stream 
indices (i.e., coded data conesponding to the objects) are 
described 

Accordingly, the decoder 204 performs decoding and 
composition of coded data of the respective objects on the 
basis of the scene description information Sf and the object 
descriptor OD, and generates a regeneration data Rg for 
displaying the scene 201. 

Furthermore, in the system 200 in FIG. 11, the objects and 
the object descriptors are defined with the video data being 
distinguished from the audio data. 
[Embodiment 5] 

An object data decoding apparatus of a fifth embodiment 
of the present invention in the object data transmission 
system 20 will now be described. 

FIG. 12 is a block diagram showing the object data 
decoding apparatus of the fifth embodiment. Note that the 
objects and the object descriptors are defined without dis- 
tinguishing the video data from the audio data in this 
embodiment. 

Referring to FIG. 12, an object data decoding apparatus 
105 is shown. The object data decoding apparatus 105 is 
used for receiving a multiplexed bit stream MEgl compris- 
ing a scene description information Sf and an object descrip- 
tor OD rather than the composition stream and the stream 
association table as the auxiliary data, and reproducing 
video data Rg of one scene from the multiplexed bit stream 
MEgl. 
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The multiplexed bit stream MEgl comprises coded data 
in which scene data of one scene 150 in FIG. 13(«) has been 
coded for each object of the scene, and the auxiliary data. 
Referring to FIGS. 13(a) and 13(b), the scene 150 com- 

5 prises plural objects (small images) of a hierarchical struc- 
ture. More specifically, the scene 150 comprises d back- 
ground image 151 as a background, a mobile 152 moving in 
the background, logo (Let's start) 153 displayed on the 
background image, and first and second wheels 154 and 155, 

10 which correspond to the objects. The background image 151 
serves AR a node, and the mobile 152 and the logo 153 
belong thereto. Also, the mobile 152 serves as a node, and 
the first and second wheels 154 and 155 belong thereto. 
Coded data of the mobile 152 comprises coded data of a 

15 window 152(1, a body 1526, and a chimney 152c. 

The auxiliary data of the multiplexed bit stream MEgl 
comprises the scene description information and the object 
descriptor. FIG, 14(d) shows a scene description SDl on the 
basis of the scene description information. 

20 The scene description SDI describes the scene 150. In 
FIG. 14(a), Object(l) 161 to 0bject(5) 165 are shown, 
which are descriptors which indicate the background image 
151, the mobile 152, the logo 153, and the first and second 
wheels 154 and 155, respectively. As is seen from these 

25 descriptors, the mobile 152 and the logo 153 belong to the 
background image 153, and the first and second wheels 154 
and 155 belong to the mobile 152, To each of the descriptors 
161 to 165, Object id (Old) by which coded data of respec- 
tive objects of the multiplexed bit stream MEgl can be 

30 identified is given ("id" indicates index). Specifically, to the 
descriptors 161 to 165, Object id (iid) 1 to 5 are given, 
respectively. 

Furthermore, in an object descriptor OD shown in FIG. 
14(6), correspondence between Object id and Stream id is 

35 shown. As shown in FIG. 14(6), Object id (Oid-1), Object 
id (Oid«2), Object id (Oid=3), and Object id (Oid=4, 5), 
correspond to Stream id (Sid-1, 2), Stream id (Sid =3 to 6), 
Stream id (Sid-7), and Stream id (Sid-8), respectively. 
Furthermore, in the object descriptor OD, data type of each 

40 object, i.e., "Video" or "Audio" is described. 

The object data decoding apparatus 105 of the fifth 
embodiment will be described hereinafter. 

Referring to FIG. 12 again, a basic construction of the 
object data decoding apparatus 105 is identical to that of the 

45 object data decoding apparatus 101 of the first embodiment. 
Specifically, the decoding apparatus 105 comprises a demul- 
tiplexer 11a for extracting the scene description information 
Sf and the object descriptor OD as auxiliary data Dsub 
included in the multiplexed bit stream MEgl, and extracting 

50 coded data Eg of each object from the multiplexed bit stream 
MEgl in accordance with a first control signal Contl, an 
audio and video (AV) decoder 12 for decoding the coded 
data Eg in accordance with a second control signal Cont2 
and outputting reproduced data Rg of each scene, and a CPU 

55 13a for deciding a stream id of the coded data Eg on the 
basis of the object descriptor OD and supplying the demul- 
tiplexer 11a with the first control signal Contl on the basis 
of the decision result, and supplying the AV decoder 12 with 
information on placement of objects of one scene and 

60 information on display start time of each object using the 
control signal Cont2 on the basis of the scene description 
information Sf. 

In this fifth embodiment, the CPU is used to create an 
object table indicating correspondence between objects and 

65 the corresponding coded data Eg (stream) on the basis of the 
scene description information Sf and the object descriptor 
OD, to be stored in a data storage means in the CPU 13a. 
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FIGS. lS{a) and 15(6) are diagrams showing an object synthesizesdecodeddataof respective objects in accordance 

table T3 of the scene 150. with the second control signal Cont2, and outputs repro- 

Referring to these figures, the object table T3 has a duced data Rg of one scene comprising plural objects, 

hierarchical structure, and comprises a main table T3« This decoding is identical to that of the prior art object 

indicating a correspondence between objects of the scene 5 decoding apparatus as already described. 

150 and the corresponding streams, and a sub table T3fe In this fifth embodiment, in addition to the decoding, the 

indicating d correspondence between video or audio of each CPU 13d! creates the object table T3 in RGS. lS(a) and 

object and the corresponding stream. lS{b). 

In these tables, various information associated with Hereinafter, creating of the object table T3 following a 

Object id is entered. lo flowchart in FIG. 16. 

Specifically, the Object id is given to each object descrip- In step Sla, the scene description information Sf is read 

tor in the scene description in FIG. 14(a) to have a one-to- to the data storage means of the CPU 13a. In step S2fl, the 

one correspondence, and entered in ascending order of value object descriptor OD is read to the data storage means of the 

"Oid" of the Object id. CPU 13a. In step S3fl, descriptors of respective objects of 

Object indices (Oid=l-5) are given to the descriptors 161 is the scene description SDI on the basis of the scene descrip- 

to 165 of objects 151 to 151, respectively. tion information Sf are loaded into an operating unit of the 

In the main table T3fl, type or each object (video or CPU 13a, and Oid of each object id is given to each 

audio), end the corresponding stream id are entered. In the descriptor, thereby an object can be identified by the corre- 

main table T3a, indices of upper and lower objects of each spending object id. 

object, stream indices of the upper and lower objects, 20 In step S6a, the Oid is entered in the object table 13 as 

common object indices of objects which share a stream, and "id". 

priorities of respective objects are also entered. In step S7a, the object descriptor OD is interpreted. On 

The upper object is in a higher-order layer than an object, the basis of the interpretation result, various table compo- 

and the lower object is in a lower-order layer than the object. nents of respective objects are entered in the object table T3, 

Specifically, since no upper object of an upper most 25 The components are the stream id, the priority, the stream 

object, i.e., the object in a first layer Lla in FIG. 13(a) exists, type (Video or Audio) of each object. At this time, as the 

Oid of the corresponding upper object id and Sid of the table components, indices of the upper and lower objects, 

corresponding upper stream id are respectively "0", Upper stream indices of the upper and lower objects, and the 

objects of objects in second and third layers L2a and L3a are common object indices are also entered in the object table 

objects in the first and second layers Lla and L2a, respec- 3D T3. 

tively. In case of a lower most object, i.e., an object having In step S9fl, it is decided whether the object which is being 

no lower object, Oid of the corresponding lower object id processed is in an upper most layer or not. 

and Sid of the corresponding lower stream id are respec- When decided in step S9a that the object is not in the 

tively "0". upper most layer, in step S12a an object layer in hierarchy 

In case of objects 151 and 152 each comprising plural 35 is raised by one, and then in step S9a, the decision step is 

pieces of video and audio, the corresponding stream types performed again. On the other hand, when decided in step 

"H" are described in the main table T3a, and stream indices S9a that the object is in the upper most layer, in step SlOa, 

of coded data of video and audio of these objects are it is decided whether table components of all objects have 

described in the sub table T3b. been entered in the object table T3 or not. 

As can be seen from the object table T3, an object having 40 When decided in step SlOa that the components have not 

Oid-1 comprises two objects, since Oid of the correspond- been entered, the CPU performs step S3fl again and the steps 

ing lower object id "2, 3'* is described, and an object having S6a, S7a, S9a, and S12a are performed. On the other hand, 

Oid=2 comprises two objects, since Oid of the correspond- when decided the components have been entered, the CPU 

ing lower object id ''4, 5" is described. 13a completes creating the object table. 

Subsequently, operation of the object data decoding appa- 45 So created object table is stored in the data storage means 

ratus 105 of the fifth embodiment will now be described. of the CPU 13a. Each time the scene description information 

Referring to FIG. 12 again, when the multiplexed bit Sf and the object descriptor OD are updated, the stored table 

stream MEgl is input to the object data decoding apparatus is also updated to describe newest information. Therefore, 

105, the demultiplexer lla extracts the scene description the object table remains unchanged unless an object of one 

information SDI and the object descriptor OD as the auxil- 50 scene is changed. 

iary data Dsub from the multiplexed bit stream MEgl and When there is a flag in a bit stream indicating that new 

outputs the Dsub to the CPU 13a. scene description information Sf and the object descriptor 

The CPU 13a recognizes a correspondence between OD have been transmitted, the object table may be updated 

objects of the scene 150 in FIG. 13(a) and the corresponding only when the flag is transmitted. 

stream indices on the basis of the object descriptor OD, and 55 Also in the object data decoding apparatus 105, the same 

outputs the first control signal Contl on the basis of the effects as provided in the object data decoding apparatus 101 

result to the demultiplexer lla. of the first embodiment are obtained. 

The demultiplexer lla collects plural pake ted streams in Although iii the fifth embodiment, the object data decod- 
the multiplexed bit stream for each object and outputs the ing apparatus has been described as the object data process- 
resulting stream to the AV decoder 12. 60 ing apparatus in the system according to MPEG4, the object 

At this time, the CPU 13a extracts information on a data selecting apparatus of the second embodiment, the 

display position and display start time of each object from object data recording apparatus of the third embodiment, the 

the scene description information SDI and outputs the object data output apparatus of the fourth embodiment can 

information to the AV decoder 12 as the second control respectively create the object table T3 from the scene 

signal Cont2. 65 description information Sf and the object descriptor OD. 

The AV decoder 12 decodes streams of respective objects In addition, in the object data decoding apparatus which 

(a series of coded data Eg) from the demuUiplexer lla, receives the multiplexed bit stream MEg output from the 
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data output apparatus, creating the object table is dispensed 
with, and therefore the same effects as in the fifth embodi- 
ment are obtained with a simple construction. 

Although in this fifth embodiment, the objects and the 
object descriptors are defined without distinguishing the 5 
video data from the audio data, they may be defined with the 
video data being distinguished from the audio data as in the 
system 200 in FIG. 11. In this case, since the data type is 
clearly shown in the scene description, it is not necessary to 
describe the data type in the object descriptor. 

Furthermore, a program which implements constructions 
of the processing apparatus and the recording apparatus is 
recorded in a data recording medium such as a floppy disc, 
whereby processings in the embodiments are carried out in 
an independent computer system with ease. This is described 
below. 1^ 

FIGS. 17(a) to 17(c) are diagrams showing signal pro- 
cessing of the object data processing apparatus and the 
object data recording apparatus of the embodiments in a 
computer system using a floppy disc which stores a program 
of the signal processing. 20 

FIG. 17(a) shows a front appearance and a cross-section 
of a floppy disc FD, and a floppy disc body D as a recording 
medium, and FIG. 17(6) shows a physical format of the 
floppy disc body D, 

Referring to FIGS. 17(a) and 17(6), the floppy disc body 25 
D is stored in a case F, and in a surface thereof, plural tracks 
Trs are formed concentrically from outer to inner radius 
thereof, each track being divided into 16 sectors Se in an 
angle direction. Data of the program is recorded in allocated 
areas on the floppy disc body D. 30 

FIG. 17(c) is a diagram showing a construction with 
which the program is recorded/reproduced in/from the 
floppy disc FD. In case of recording the program in the 
floppy disc FD, data of the program is written thereto 
through the floppy disc drive FDD from the computer 35 
system Cs. In another case of constructing the image trans- 
mission method or image decoding apparatus in the com- 
puter system Cs using the program in the floppy disc FD, the 
program is read from the floppy disc FD by means of the 
floppy disc drive FDD and transferred to the computer 40 
system Cs. 

Although image processing ill the computer system using 
the floppy disc as the data recording medium has been 
described, this image processing is implemented using an 
optical disc. Further, the recording medium is not limited 45 
thereto, and IC card, ROM cassette, or the like may be used 
so long as it can record a program. 

Although the data recording medium which stores the 
program of transmission or decoding in the embodiments 
has been described, it may store the multiplexed bit stream 50 
MEg or non -multiplexed bit stream in the embodiments. The 
data storage means of the recording apparatus of the third 
embodiment may be realized using the data recording 
medium in FIGS. 17(a) to 17(c). 

What is claimed is: 55 

1. An object data processing apparatus for decoding N 
pieces of coded stream data (N«positive integer) obtained by 
compressively coding N pieces of object data which con- 
stitute individual data to be recorded or transmitted and have 
a hierarchical stmcture, for each object data, said apparatus gQ 
including: 

hierarchical information extraction means for extracting 
hierarchical information showing the hierarchical rela- 
tionship of the N pieces of object data, according to the 
coded stream data; and 65 

table creation means for creating, according to the hier- 
archical information, an object table including first 
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identifiers for identifying the respective object data and 
second identifiers for identifying coded stream data, 
corresponding to the respective object data, and show- 
ing a correlation between the respective second iden- 
tifiers and the corresponding first identifiers as wefl as 
the hierarchical relationship of the object data. 

2. An object data processing apparatus for decoding N 
pieces of coded stream data (N=positive integer) obtained by 
compressively coding scene data corresponding to one 
scene, for each of N pieces of objects constituting the scene, 
said apparatus including: 

hierarchical information extracting means for extracting 
hierarchical information showing the hierarchical rela- 
tionship of the respective objects constituting the scene, 
according to the coded stream data; and 

table creation means for creating, according to the hier- 
archical information, an object table including first ' 
identifiers for identifying the respective objects and 
second identifiers for identifying coded stream data, 
corresponding to the respective objects, and showing a 
correlation between the respective second identifiers 
and the corresponding first identifiers as well as the 
hierarchical relationship of the object data, 

3. The object data processing apparatus of claim 2 
wherein: 

said hierarchical information extraction means is con- 
structed so that it extracts priority information showing 
the priority order of the respective objects, according to 
the coded stream data, in addition to the hierarchical 
information; and 

said tabic creation means is constructed so that it creates, 
according to the hierarchical information and the pri- 
ority information, an object table including first iden- 
tifiers for identifying the respective objects, second 
identifiers for identifying coded stream data, corre- 
sponding to the respective objects, and priority infor- 
mation for showing the priority order of the respective 
objects, and showing a correlation between the respec- 
tive second identifiers and the corresponding first iden- 
tifiers as well as the hierarchical relationship of the 
object data. 

4. The object data processing apparatus of claim 2 further 
including: 

identification information detection means for detecting 
second identifiers correlated with first identifiers for 
identifying coded stream data of a specific object 
designated, with reference to the object table; and 

decoding means for extracting coded stream data of the 
specific object from the N pieces of coded stream data 
according to the second identifiers, and decoding the 
extracted coded strewn data, 

5. An object data processing apparatus for decoding 
multiplexed data including N pieces of coded stream data 
(N«positive integer) obtained by compressively coding 
scene data corresponding to one scene, for each of N pieces 
of objects constituting the scene, said apparatus including: 

hierarchical information extraction means for extracting 
hierarchical information showing the hierarchical rela- 
tionship of the N pieces of objects constituting the 
scene, according to information showing the correla- 
tion of the respective coded stream data and included in 
the multiplexed data; and 

table creation means for creating, according to the hier- 
archical information, an object table including first 
identifiers for identifying the respective objects and 
second identifiers for identifying coded stream data. 
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corresponding to the respective objects, and showing a 
correlation between the respective second identifiers 
and the corresponding first identifiers as well as the 
hierarchical relationship of the object data. 

6. The object data processing apparatus of claim 5 
wherein: 

said hierarchical information extraction means is con- 
structed so that it extracts priority information showing 
the priority order of the respective objects, according to 
the multiplexed data, in addition to the hierarchical 
information; and 

said table creation means is constructed so that it creates, 
according to the hierarchical information and the pri- 
ority information, an object table including first iden- 
tifiers for identifying the respective objects, second 
identifiers for identifying coded stream data corre- 
sponding to the respective objects, and priority infor- 
mation for showing the priority order of the respective 
objects, and showing a correlation between the respec- 
tive second identifiers and the corresponding first iden- 
tifiers as well as the hierarchical relationship of the 
object data. 

7. TTie object data processing apparatus of claim 5 further 
including: 

identification information detection means for detecting 
second identifiers correlated with first identifiers for 
identifying coded stream data of a specific object 
designated, with reference to the object table; and 

decoding means for extracting coded stream data of the 
specific object from the multiplexed data according to 
the second identifiers, and decoding the extracted 
coded stream data. 

8. An object data processing apparatus for processing 
multiplexed data including N pieces of coded stream data 
(N=positive integer) and being partitioned into plural pack- 
ets each having a prescribed code quantity, which coded 
stream data are obtained by compressively coding N pieces 
of object data which constitute individual data to be 
recorded or transmitted and have a hierarchical structure, for 
each object data, said apparatus including: 
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hierarchical information extraction means for extracting 
hierarchical information showing the hierarchical rela- 
tionship of the N pieces of object data, according to 
information showing the correlation of the respective 
coded stream data and included in the multiplexed data; 
and 

table creation means for creating, according to the hier- 
archical information, an object table including first 
identifiers for identifying the respective objects and 
second identifiers for identifying coded stream data, 
corresponding to the respective objects, and showing a 
correlation between the respective second identifiers 
and the corresponding first identifiers as well as the 
hierarchical relationship of the plural packets consti- 
tuting the multiplexed data. 
9. An object data processing apparatus for processing 
multiplexed data including N pieces of coded stream data 
(N=positive integer) and being partitioned into plural pack- 
ets each having a prescribed code qfuantity, which coded 
stream data are obtained by compressively coding scene data 
corresponding to one scene, for each of N pieces of objects 
constituting the scene, said apparatus including: 

hierarchical information extraction means for extracting 
hierarchical information showing the hierarchical rela- 
tionship of the respective objects constituting the scene, 
according to information showing the correlation of the 
respective coded stream data included in the multi- 
plexed data; and 
table creation means for creating, according to the hier- 
archical information, an object table including first 
identifiers for identifying the respective objects and 
second identifiers for identifying coded stream data, 
corresponding to the respective objects, and showing a 
correlation between the respective second identifiers 
with the corresponding first identifiers as well as the 
hierarchical relationship of the plural packets consti- 
tuting the multiplexed data. 
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